Classroom Observation and Value-Added Models Give Complementary Information about Quality of Mathematics Teaching

We developed the UTeach Observation Protocol (UTOP), which provides a systematic way to organize observations about teachers and students in a classroom, and provides numerical ratings of classroom quality in multiple dimensions. Through the Measures of Effective Teaching project we obtained UTOP ratings and comments on 982 videos of grades 4-8 mathematics classrooms. We also obtained results for each teacher in the videos from value-added models, which use changes in student test scores to evaluate teachers. We studied the connections between the UTOP ratings and the value-added model ratings. We were surprised by many findings. For example, the particular classroom attributes that lead to the largest student test score gains at sixth grade lead to the lowest test score gains at fifth grade. Our main conclusion is that classroom observation and value-added models supply complementary and separately valuable information on what happens in classrooms. Neither one nor the other can be used in isolation, nor does averaging the results together retain enough information. In the best classrooms, both observation results and student test-score gains are favorable. Note: This material will appear as a chapter in a research volume on the Measures of Effective Teaching project. It is reproduced with permission of John Wiley & Sons Inc. We are grateful to the Bill and Melinda Gates Foundation for funding that enabled us to carry out this work. Partial funding for the development and pilot studies of the UTOP were provided by the National Science Foundation through a Noyce Scholarships grant, DUE-0630376. Inclusion of the UTOP in the MET project was organized by the National Math and Science Initiative in cooperation with Laying the Foundations, partially funded by the Carnegie Corporation. We particularly thank John Winn, Matthew Valerius, and Christy Hovanetz for their assistance. We also thank members of the original UTOP pilot team, including: Mary Walker, Larry Abraham, Prerna Arora, Shasta Ihorn, and Jessica Gordon. The opinions expressed in this chapter are the responsibility of the authors and are not necessarily shared by the National Science Foundation, Bill and Melinda Gates Foundation, National Math and Science Initiative, or Carnegie Corporation.